Skip to main content

Inside Box

Captures whether a user's body or joints are inside a set visual region on the camera. By default the skeleton is not drawn to allow users to customize their experience fully, see Overlay docs to add an overlay

Inside Box

Feature​

Draws a box for the user to enter. Note the box is draw relative to the camera edges, not the view screen.

let edgeInsets = QuickPose.RelativeCameraEdgeInsets(top: 0.2, left: 0.2, bottom: 0.2, right: 0.2)
case inside(edgeInsets) // default whole body and default style
case inside(edgeInsets, limb: .head) // just the head, and default style
case inside(edgeInsets, limb: .head, style: customStyle) // just the head, and a custom style

This feature is styled by default as a red box which changes to green when the user is in the highlighted box.

QuickPose.Style(cornerRadius: 12, 
                color: UIColor.red,
                conditionalColors: [
                    QuickPose.Style.ConditionalColor(min: 1, max: nil, color: UIColor.green)
                    ]
                ) // red, but change to green if value is 1.

Basic Implementation​

To show results you'll need to modify your view ZStack, which is we assume is setup as described in the Getting Started Guide:

ZStack(alignment: .top) {
    QuickPoseCameraView(useFrontCamera: true, delegate: quickPose)
    QuickPoseOverlayView(overlayImage: $overlayImage)
}

The basic implementation will require displaying some text to the screen, start with declaring this value in your swiftui view.

@State private var feedbackText: String? = nil

And show this feedback text as an overlay to the view in your branding.

ZStack(alignment: .top) {
    QuickPoseCameraView(useFrontCamera: true, delegate: quickPose)
    QuickPoseOverlayView(overlayImage: $overlayImage)
}
.overlay(alignment: .center) {
    if let feedbackText = feedbackText {
        Text(feedbackText)
            .font(.system(size: 26, weight: .semibold)).foregroundColor(.white).multilineTextAlignment(.center)
            .padding(16)
            .background(RoundedRectangle(cornerRadius: 8).foregroundColor(Color("AccentColor").opacity(0.8)))
            .padding(.bottom, 40)
    }
}

Note the above use of alignment in .overlay(alignment: .center), you can modify this to move the overlay around easily to say the bottom: .overlay(alignment: .bottom).

For this basic version it fills the feedback text with the Inside Box result as a percentage, and hides the text when the feature result is not available.

quickPose.start(features: [.inside(edgeInsets)], onFrame: { status, image, features, feedback, landmarks in
    switch status {
        case .success:
            overlayImage = image
            if let result = features.values.first  {
                feedbackText = result.stringValue
            } else {
                feedbackText = nil
            }
        case .noPersonFound:
            feedbackText = "Stand in view";
        case .sdkValidationError:
            feedbackText = "Be back soon";
    }
})

Conditional Styling​

To give user feedback consider using conditional styling so that when the user's measurement goes above a threshold, here 0.8, a green highlight is shown.

let greenHighlightStyle = QuickPose.Style(conditionalColors: [QuickPose.Style.ConditionalColor(min: 0.8, max: nil, color: UIColor.green)])
quickPose.start(features: [.inside(edgeInsets, style: customOrConditionalStyle)], 
                onFrame: { status, image, features, feedback, landmarks in  ...
})

Improving the Captured Results​

The basic implementation above would likely capture an incorrect value, as in the real world users need time to understand what they are doing, change their mind, or QuickPose can simply get an incorrect value due to poor lighting or the user's stance. These issues are partially mitigated by on-screen feedback, but it's best to use an QuickPoseDoubleUnchangedDetector to keep reading the values until the user has settled on a final answer.

To steady the .inside(edgeInsets) results declare a configurable Unchanged detector, which can be used to turn lots of our input features to read more reliably. 

@State private var unchanged = QuickPoseDoubleUnchangedDetector(similarDuration: 2)

This will on trigger the callback block when the result has stayed the same for 2 seconds, the above has the default leniency, but this can be modified in the constructor.

@State private var unchanged = QuickPoseDoubleUnchangedDetector(similarDuration: 2, leniency: Double = 0.2) // changed to 20% leniency

The unchanged detector is added to your onFrame callback, and is updated every time a result is found, triggering its onChange callback only when the result has not changed for the specified duration.

quickPose.start(features: [.inside(edgeInsets)], onFrame: { status, image, features, feedback, landmarks in                
    switch status {
        case .success:
            overlayImage = image
            if let result = features.values.first  {
                feedbackText = result.stringValue
                unchanged.count(result: result.value) {
                    print("Final Result \(result.value)") 
                    // your code to save result
                }
            } else {
                feedbackText = nil // blank if no hand detected
            }
        case .noPersonFound:
            feedbackText = "Stand in view";
        case .sdkValidationError:
            feedbackText = "Be back soon";
    }
})